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WHAT IS CLAIMED AND DESIRED TO BE SECURED BY LETTERS 
PATENT OF THE UNITED STATES IS : 



1 1, A method, comprising the step of: 

2 warping a set of images synchronously captured 

3 from a camera array into a common coordinate system of 

4 a composite image, 

1 'I. The method according to claim 1, wherein said 

2 step of warping comprises the step of : 

3 applying a transformative equation to selected 

4 pixels of each image to transform the selected pixels 

5 from a coordinate system of the image to said common 

6 coordinate system. 

1 3 . The method according to claim 2 , wherein said 

2 transformative equation is at least one of a bilinear 

3 transformation, radial transformation, perspective 

4 transformation, and affine transformation. 

1 4. The method according to claim 1, wherein said 

2 step of warping comprises the steps of: 

3 identifying a set of contiguous patches in each of 

4 said images; and 
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5 warping each patch into said common coordinate 

6 system. 

1 5. The method according to claim 4, wherein said 

2 step of identifying patches, comprises: 

3 identifying regions in each image having 

4 registration points in common with said composite 

5 image . 



1 6, The method according to claim 4, wherein said 

2 step of warping, includes the step of: 

3 combining patches from different images having a 

4 same set of registration points into a same 

5 corresponding location of said composite image. 

1 7. The method according to claim 6, wherein said 

2 step of combining comprises the step of: 

3 cross -fading patches from different images having 

4 a same set of registration points into a same 

5 corresponding location of said composite image. 
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8. The method according to claim 6, wherein said 




fading, blurring, averaging, (and other image ef fec^E^to 



seamlessly combine said patches from different images 



Attorney Docket No.: FXPL- 01003US0 MCF/JWC 
jwc/fxpl/1003/1003 . 001 



Express Mail #EL328295759US 



# 



-57- 

5 having a same set of registration points into a same 

6 corresponding location of said composite image, 

1 9 . The method according to claim 4 , wherein said 

2 step of warping each patch comprises the step of: 

3 performing a transformation on each patch to 

4 transform pixels of each respective patch from a 

5 coordinate system of the respective patch to said 

6 common coordinate system. 

1 10, The method according to claim 9, wherein said 

2 step of performing a transformation comprises: 

3 applying a predetermined bilinear transformation 

4 matrix to each patch, said predetermined bilinear 

5 transformation matrix representing an amount of warping 

6 required for each patch to transform each patch into 

7 said common coordinate system, 

1 11. The method according to claim 7, wherein said 

2 step of cross -fading comprises the steps of: 

3 varying a parameter of each pixel of a first patch 

4 from minimum value at a first boundary of said first 

5 patch to a maximum value at an opposite boundary of 

6 said first patch; 
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7 varying said parameter of each pixel of a second 

8 patch from said maximum value at a boundary of said 

9 second patch corresponding to said first boundary to 

10 said minimum value at a boundary of said second patch 

11 corresponding to said opposite boundary; 

12 summing each of corresponding pixel values of each 

13 said first patch and said second patch; and 

14 placing the summed values in corresponding 

15 locations of said common coordinate system. 

1 12. The method according to claim 11, wherein 

2 said parameter is at least one of brightness, contrast, 

3 and intensity. 

1 13. The method according to claim 1, further 

2 comprising the step of: 

3 repeating said step of warping for each set of 

4 images synchronously captured by said camera array, 

5 each set of images representing one frame in a video 

6 stream of said scene. 

1 14. The method according to claim l, further 

2 comprising the steps of: 

3 selecting an area of interest from the composite 

4 image; and 
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5 outputting the selected area of interest to a 

6 user, 

1 15. The method according to claim 14, wherein 

2 said step of outputting comprises the step of 

3 displaying said area of interest from the combined 

4 warped images . 

1 16. The method according to claim 14, wherein 

2 said step of selecting comprises: 

3 directing said area of interest to a predetermined 

4 area surrounding at least one of motion detected in 

5 said scene, audio sources detected within said scene, 

6 and proximity of objects detected in said scene. 

1 17. A method of controlling a virtual camera 

2 having a view selected from an array of cameras, 

3 comprising the steps of: 

4 combining images from each of said cameras into a 

5 panoramic view; 

6 detecting motion in said panoramic view; and 

7 directing a view of said virtual camera based on 

8 said motion. 
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1 18, The method according to claim 17, wherein: 

2 said method further comprises the step of: 

3 ranging objects in said panoramic view by 

4 stereoscopically calculating a range of said objects; 

5 and 

6 said step of directing comprises directing a view 

7 of said virtual camera based on the motion detected and 

8 a range of said objects, 

1 19. The method according to claim 18, wherein: 

2 said method further comprises the steps of: 

3 retrieving audio inputs from a set of directive 

4 microphones, and 

5 determining a direction of said audio inputs 

6 within said panoramic view; and 

7 said step of directing comprises directing a view 

8 of said virtual camera based on at least one of said 

9 motion, a range of said objects, and a direction of 
10 said audio inputs. 

1 20. A camera array, comprising: 

2 a set of cameras mounted in an array; 

3 an image combining mechanism configured to combine 

4 at least two of images captured from said set of 

5 cameras into a composite image; 
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6 a view selection device configured to select a 

7 view from the composite image; and 

8 an output mechanism configured to display the 

9 selected view. 



1 21, The camera array according to claim 20, 

2 wherein: 

3 said image combining mechanism includes, 

4 a warping device configured to warp patches of 

5 each image into a common coordinate system of said 

6 composite image, and 

7 a fading device configured to fade and combine 

8 patches having a same location in said composite image. 

1 22, The camera array according to claim 20, 

2 wherein said frame selection device includes: 

3 at least one of, 

4 a video motion detector configured to detect 

5 motion in the composite image, 

6 a sound detection device configured to determine 

7 a location of sound originating within said composite 

8 image , and 

9 a stereoscopic ranging mechanism configured to 
10 utilize at least two images from separate cameras of 
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11 said camera array to determine a range of objects in 

12 said composite image; and 

13 a detection mechanism configured to automatically 

14 detect any of a face, shape, color, and motion of a 

15 subject for inclusion in said selected frame based on 

16 at least one of ranging data from said stereoscopic 

17 ranging mechanism, location data from said sound 

18 detection device, and motion detected by said video 

19 motion detector. 

1 23. The camera array according to claim 21, 

2 wherein: 

3 said array of cameras are immovably mounted on a 

4 firm fixed base; and 

5 said warping device applies a pre -determined 

6 transformation to each of said patches. 

1 24. The camera array according to claim 21, 

2 further comprising a registration mechanism configured 

3 to register each of said camera arrays by finding 

4 registration points in common with views of each camera 

5 and said composite image. 

1 25. The camera array according to claim 24, 

2 wherein said registration device comprises: 
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3 a registration point source directable to 

4 locations within each camera view; and 

5 a detection mechanism configured to detect said 

6 registration point source and register the detected 

7 registration point source in each of said camera views 

8 with respect to a coordinate system of said composite 

9 image . 

1 26, The camera array according to claim 25, 

2 wherein said registration point source is at least one 

3 of a light source directed into view of said camera 

4 array and a grid placed in view of said camera array. 

1 2 7. A method of registering a camera array, 

2 comprising the steps of: 

3 placing at least one registration point in a field 

4 of view of at least two cameras of said camera array; 

5 identifying a location of each registration point 

6 in a field of view of each camera of said array; and 

7 maintaining information about each registration 

8 point in relation to each camera such that images may 

9 be combined in relation to said registration points. 

1 28, The method according to claim 27, wherein 

2 said step placing comprises the steps of: 
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3 sweeping an beam light source across a field of 

4 view of at least two cameras of said camera array • 

1 29, The method according to claim 27, wherein 

2 said step of placing comprises the step of: 

3 placing a grid of known location in a field of 

4 view in at least two cameras of said array, 

1 30, The method according to claim 27, wherein 

2 said step of maintaining information comprises the 

3 steps of: 

4 calculating a warped coordinate system for placing 

5 pixels of said cameras; 

6 maintaining a table identifying pixels of said 

7 cameras, an amount of fade, and a corresponding 

8 location in the warped coordinate system. 
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